{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# XGBoost Parameter Tuning for Rent Listing Inqueries"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第四步：调整树的参数：subsample 和 colsample_bytree\n",
    "(粗调，参数的步长为0.1；下一步是在粗调最佳参数周围，将步长降为0.05，进行精细调整)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先 import 必要的模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from xgboost import XGBClassifier\n",
    "import xgboost as xgb\n",
    "\n",
    "import pandas as pd \n",
    "import numpy as np\n",
    "\n",
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.model_selection import StratifiedKFold\n",
    "\n",
    "from sklearn.metrics import log_loss\n",
    "\n",
    "from matplotlib import pyplot\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# path to where the data lies\n",
    "dpath = './data/'\n",
    "train = pd.read_csv(dpath +\"RentListingInquries_FE_train.csv\")\n",
    "test = pd.read_csv(dpath +\"RentListingInquries_FE_test.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "每类样本分布不是很均匀，所以交叉验证时也考虑各类样本按比例抽取"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "Y = train['interest_level']\n",
    "X = train.drop([\"interest_level\"], axis=1)\n",
    "X = np.array(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train=X\n",
    "y_train=Y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "各类样本不均衡，交叉验证是采用StratifiedKFold，在每折采样时各类样本按比例采样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# prepare cross validation\n",
    "kfold = StratifiedKFold(n_splits=5, shuffle=True, random_state=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "第二轮参数调整得到的n_estimators最优值（231），其余参数继续默认值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "用交叉验证评价模型性能时，用scoring参数定义评价指标。评价指标是越高越好，因此用一些损失函数当评价指标时，需要再加负号，如neg_log_loss，neg_mean_squared_error 详见sklearn文档：http://scikit-learn.org/stable/modules/model_evaluation.html#log-loss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'colsample_bytree': [0.6, 0.7, 0.8, 0.9],\n",
       " 'subsample': [0.3, 0.4, 0.5, 0.6, 0.7, 0.8]}"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#max_depth 建议3-10， min_child_weight=1／sqrt(ratio_rare_event) =5.5\n",
    "subsample = [i/10.0 for i in range(3,9)]\n",
    "colsample_bytree = [i/10.0 for i in range(6,10)]\n",
    "param_test = dict(subsample=subsample, colsample_bytree=colsample_bytree)\n",
    "param_test\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\model_selection\\_search.py:761: DeprecationWarning: The grid_scores_ attribute was deprecated in version 0.18 in favor of the more elaborate cv_results_ attribute. The grid_scores_ attribute will not be available from 0.20\n",
      "  DeprecationWarning)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "([mean: -0.59064, std: 0.00445, params: {'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  mean: -0.58690, std: 0.00383, params: {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  mean: -0.58572, std: 0.00302, params: {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  mean: -0.58398, std: 0.00406, params: {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  mean: -0.58333, std: 0.00360, params: {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  mean: -0.58232, std: 0.00334, params: {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  mean: -0.58968, std: 0.00321, params: {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  mean: -0.58655, std: 0.00392, params: {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  mean: -0.58459, std: 0.00288, params: {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  mean: -0.58378, std: 0.00317, params: {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  mean: -0.58249, std: 0.00351, params: {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  mean: -0.58253, std: 0.00337, params: {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  mean: -0.58918, std: 0.00355, params: {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  mean: -0.58739, std: 0.00356, params: {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  mean: -0.58500, std: 0.00358, params: {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  mean: -0.58331, std: 0.00369, params: {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  mean: -0.58282, std: 0.00332, params: {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  mean: -0.58222, std: 0.00338, params: {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  mean: -0.58990, std: 0.00247, params: {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  mean: -0.58682, std: 0.00351, params: {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  mean: -0.58506, std: 0.00373, params: {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  mean: -0.58348, std: 0.00358, params: {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  mean: -0.58285, std: 0.00370, params: {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  mean: -0.58287, std: 0.00339, params: {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       " -0.582215264335301)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgb = XGBClassifier(\n",
    "        learning_rate =0.1,\n",
    "        n_estimators=231,  #第二轮参数调整得到的n_estimators最优值\n",
    "        max_depth=6,\n",
    "        min_child_weight=7,\n",
    "        gamma=0,\n",
    "        subsample=0.3,\n",
    "        colsample_bytree=0.8,\n",
    "        colsample_bylevel = 0.7,\n",
    "        objective= 'multi:softprob',\n",
    "        seed=3)\n",
    "\n",
    "\n",
    "gsearch = GridSearchCV(xgb, param_grid = param_test, scoring='neg_log_loss',n_jobs=-1, cv=kfold)\n",
    "gsearch.fit(X_train , y_train)\n",
    "\n",
    "gsearch.grid_scores_, gsearch.best_params_,     gsearch.best_score_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "{'mean_fit_time': array([463.40162086, 519.93637605, 548.90441184, 549.04791656,\n",
       "        544.16693649, 542.07845874, 500.23749046, 564.883605  ,\n",
       "        613.08328323, 618.61419163, 602.2831625 , 603.83248243,\n",
       "        536.21827469, 611.8342618 , 655.39851294, 668.03040552,\n",
       "        704.63730764, 719.40717397, 654.97615571, 771.60058889,\n",
       "        843.76334224, 858.75795197, 801.54592843, 737.28439298]),\n",
       " 'mean_score_time': array([1.69560924, 1.58712087, 1.49276915, 1.51743536, 1.50409989,\n",
       "        1.57198086, 1.53718781, 1.60125842, 1.60075674, 1.57358494,\n",
       "        1.53097134, 1.51733532, 1.51502895, 1.53036985, 1.49096513,\n",
       "        1.50440035, 1.7098485 , 1.76108341, 1.78905754, 1.66703329,\n",
       "        1.76680017, 1.77963238, 1.57839699, 1.33056364]),\n",
       " 'mean_test_score': array([-0.590636  , -0.58689878, -0.58571554, -0.58397666, -0.58333296,\n",
       "        -0.58232358, -0.58968373, -0.58654747, -0.58459206, -0.58377891,\n",
       "        -0.58248967, -0.58253221, -0.58917554, -0.58739412, -0.58500258,\n",
       "        -0.58330981, -0.58282436, -0.58221526, -0.58990023, -0.58682117,\n",
       "        -0.58506043, -0.58348288, -0.58285086, -0.58287236]),\n",
       " 'mean_train_score': array([-0.49500378, -0.48850141, -0.48407512, -0.48057202, -0.47820321,\n",
       "        -0.47835269, -0.49127872, -0.48415721, -0.48037568, -0.47743397,\n",
       "        -0.476     , -0.47544194, -0.48828025, -0.48183179, -0.47762693,\n",
       "        -0.47441806, -0.47293732, -0.47367962, -0.4854159 , -0.47926709,\n",
       "        -0.47547807, -0.47219807, -0.47073577, -0.47197273]),\n",
       " 'param_colsample_bytree': masked_array(data=[0.6, 0.6, 0.6, 0.6, 0.6, 0.6, 0.7, 0.7, 0.7, 0.7, 0.7,\n",
       "                    0.7, 0.8, 0.8, 0.8, 0.8, 0.8, 0.8, 0.9, 0.9, 0.9, 0.9,\n",
       "                    0.9, 0.9],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'param_subsample': masked_array(data=[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.3, 0.4, 0.5, 0.6, 0.7,\n",
       "                    0.8, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.3, 0.4, 0.5, 0.6,\n",
       "                    0.7, 0.8],\n",
       "              mask=[False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False,\n",
       "                    False, False, False, False, False, False, False, False],\n",
       "        fill_value='?',\n",
       "             dtype=object),\n",
       " 'params': [{'colsample_bytree': 0.6, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.6, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.7, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.8, 'subsample': 0.8},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.3},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.4},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.5},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.6},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.7},\n",
       "  {'colsample_bytree': 0.9, 'subsample': 0.8}],\n",
       " 'rank_test_score': array([24, 19, 16, 12,  9,  2, 22, 17, 13, 11,  3,  4, 21, 20, 14,  8,  5,\n",
       "         1, 23, 18, 15, 10,  6,  7]),\n",
       " 'split0_test_score': array([-0.58418045, -0.58076592, -0.58096554, -0.57855021, -0.57705993,\n",
       "        -0.57770549, -0.58610162, -0.58096821, -0.58026816, -0.57997272,\n",
       "        -0.57692305, -0.57781143, -0.58395419, -0.58270368, -0.5815984 ,\n",
       "        -0.57879073, -0.57864135, -0.57742299, -0.58514752, -0.58101189,\n",
       "        -0.57908971, -0.57719565, -0.57739255, -0.57822159]),\n",
       " 'split0_train_score': array([-0.49684186, -0.49011024, -0.48661614, -0.4815401 , -0.47923458,\n",
       "        -0.47790048, -0.49292673, -0.48577067, -0.48110417, -0.47778843,\n",
       "        -0.47575086, -0.4767895 , -0.48932325, -0.48376354, -0.47937676,\n",
       "        -0.47582316, -0.47397153, -0.47521484, -0.48941395, -0.4818829 ,\n",
       "        -0.4772018 , -0.4725154 , -0.47031358, -0.47324339]),\n",
       " 'split1_test_score': array([-0.58836026, -0.58575335, -0.58459019, -0.58173767, -0.58262676,\n",
       "        -0.57993717, -0.58887786, -0.58505308, -0.58403953, -0.58147534,\n",
       "        -0.58098483, -0.5815244 , -0.58745944, -0.58581167, -0.58143867,\n",
       "        -0.58146027, -0.58145694, -0.58150392, -0.5907036 , -0.58694961,\n",
       "        -0.58459358, -0.58310287, -0.58068052, -0.58166524]),\n",
       " 'split1_train_score': array([-0.4931447 , -0.48720705, -0.48316686, -0.47986561, -0.47874425,\n",
       "        -0.47840172, -0.49029469, -0.48269536, -0.47919556, -0.47601363,\n",
       "        -0.4763186 , -0.4748452 , -0.48735406, -0.48208669, -0.47918189,\n",
       "        -0.47423517, -0.47318646, -0.4725949 , -0.48472839, -0.47917926,\n",
       "        -0.4767141 , -0.47432015, -0.47304337, -0.47310627]),\n",
       " 'split2_test_score': array([-0.58980927, -0.58624686, -0.58511467, -0.5825744 , -0.58354009,\n",
       "        -0.58182626, -0.58764897, -0.58511132, -0.58343582, -0.58263437,\n",
       "        -0.58254179, -0.58085295, -0.58830162, -0.58560664, -0.58346246,\n",
       "        -0.58097189, -0.58059405, -0.58041309, -0.59060241, -0.58560468,\n",
       "        -0.58390791, -0.58316306, -0.58254748, -0.58151357]),\n",
       " 'split2_train_score': array([-0.49607674, -0.48883978, -0.48382617, -0.48228681, -0.47848466,\n",
       "        -0.47796638, -0.49252048, -0.48348342, -0.48106434, -0.48040375,\n",
       "        -0.47748976, -0.47741304, -0.49023224, -0.48278847, -0.47704781,\n",
       "        -0.47469606, -0.47296997, -0.47376336, -0.48614205, -0.47716668,\n",
       "        -0.4751953 , -0.4715004 , -0.46988728, -0.47189593]),\n",
       " 'split3_test_score': array([-0.59369149, -0.58972875, -0.58848374, -0.58693834, -0.58570282,\n",
       "        -0.58562051, -0.59031945, -0.58921947, -0.58637274, -0.58624833,\n",
       "        -0.58461741, -0.58497715, -0.59225597, -0.59020066, -0.58852915,\n",
       "        -0.58703774, -0.5861518 , -0.58449302, -0.59066802, -0.58910283,\n",
       "        -0.5875123 , -0.5869421 , -0.58570452, -0.5846837 ]),\n",
       " 'split3_train_score': array([-0.49401399, -0.48840284, -0.4840281 , -0.47856731, -0.47788544,\n",
       "        -0.47909501, -0.49004958, -0.48386234, -0.47995665, -0.47632335,\n",
       "        -0.47496913, -0.47333065, -0.48671411, -0.47968631, -0.47601106,\n",
       "        -0.47467874, -0.47342059, -0.47436463, -0.48348962, -0.4793828 ,\n",
       "        -0.473311  , -0.47194515, -0.47034061, -0.46941161]),\n",
       " 'split4_test_score': array([-0.59714052, -0.59200058, -0.5894247 , -0.59008454, -0.58773655,\n",
       "        -0.58652977, -0.59547249, -0.59238705, -0.58884534, -0.58856526,\n",
       "        -0.58738276, -0.58749661, -0.59390793, -0.59264956, -0.58998572,\n",
       "        -0.58828993, -0.58727903, -0.58724483, -0.59238038, -0.59143823,\n",
       "        -0.5902002 , -0.58701179, -0.58793075, -0.58827933]),\n",
       " 'split4_train_score': array([-0.4949416 , -0.48794714, -0.48273834, -0.48060028, -0.4766671 ,\n",
       "        -0.47839986, -0.49060211, -0.48497427, -0.48055768, -0.4766407 ,\n",
       "        -0.47547165, -0.47483132, -0.48777757, -0.48083395, -0.47651712,\n",
       "        -0.47265715, -0.47113802, -0.47246039, -0.4833055 , -0.47872381,\n",
       "        -0.47496817, -0.47070926, -0.47009399, -0.47220647]),\n",
       " 'std_fit_time': array([ 1.31683234,  5.7695629 ,  4.50265001,  5.09448388,  2.26842308,\n",
       "         1.45064564,  2.77896545,  0.64586693,  3.28830184, 10.87067604,\n",
       "         4.14826816, 15.29321574,  2.76072011,  3.74634905,  3.46240168,\n",
       "         1.82174906, 14.10304062, 28.00039388, 22.27864625, 37.71110146,\n",
       "        31.20316796, 26.50415742, 36.05807382, 41.20141685]),\n",
       " 'std_score_time': array([0.09528561, 0.04055691, 0.05792413, 0.04456338, 0.02493845,\n",
       "        0.06034001, 0.03572433, 0.07760695, 0.05302963, 0.05953607,\n",
       "        0.04054847, 0.01430205, 0.01739865, 0.06868618, 0.0079333 ,\n",
       "        0.04170452, 0.10208081, 0.09201358, 0.10345412, 0.11571414,\n",
       "        0.11198857, 0.09359081, 0.06477943, 0.07375439]),\n",
       " 'std_test_score': array([0.00445364, 0.003831  , 0.00302024, 0.00406195, 0.00360273,\n",
       "        0.00334152, 0.00321023, 0.00391557, 0.00288432, 0.00316565,\n",
       "        0.00351342, 0.00336869, 0.0035455 , 0.00355503, 0.00357566,\n",
       "        0.00368794, 0.0033243 , 0.00338231, 0.00246849, 0.00351467,\n",
       "        0.00373151, 0.00358317, 0.00370245, 0.00338951]),\n",
       " 'std_train_score': array([0.00133937, 0.00096903, 0.00135151, 0.00129587, 0.00088275,\n",
       "        0.00042647, 0.00119957, 0.00109076, 0.00072199, 0.00160158,\n",
       "        0.00086311, 0.00147565, 0.00130088, 0.00143601, 0.00138981,\n",
       "        0.00102507, 0.00095956, 0.00104846, 0.00224271, 0.00152121,\n",
       "        0.00138095, 0.00121418, 0.00116544, 0.00137981])}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gsearch.cv_results_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('mean_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('std_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split0_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split1_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split2_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best: -0.582215 using {'colsample_bytree': 0.8, 'subsample': 0.8}\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split3_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\utils\\deprecation.py:122: FutureWarning: You are accessing a training score ('split4_train_score'), which will not be available by default any more in 0.21. If you need training scores, please set return_train_score=True\n",
      "  warnings.warn(*warn_args, **warn_kwargs)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBo\ndHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzs3Xd8jlcbwPHfyZCIkYgESazYI8te\nRSNG1V5Ram9qtS8V1Ci1Ss0aVbW1CK1dpMTeNIJEQxIjQkQiRHbynPePJ9JohhCR4Xw/n+fzyvOc\nc+5zP/W6cu5z39clpJQoiqIoytvSye4JKIqiKLmbCiSKoihKpqhAoiiKomSKCiSKoihKpqhAoiiK\nomSKCiSKoihKpqhAoiiKomSKCiSKoihKpqhAoiiKomSKXnZP4H0wMzOTZcuWze5pKIqi5CqXL19+\nIqU0f127DyKQlC1blkuXLmX3NBRFUXIVIcTdjLRTl7YURVGUTFGBRFEURckUFUgURVGUTPkg9kgU\nJSeIi4sjICCA6Ojo7J6KorzC0NCQkiVLoq+v/1b9VSBRlPckICCAQoUKUbZsWYQQ2T0dRQFASklI\nSAgBAQFYW1u/1Rjq0paivCfR0dEULVpUBRElRxFCULRo0UytlFUgUZT3SAURJSfK7N9LFUjSIKVk\n+8X7uHkFZfdUFEVRcjS1R5KGeI1k47k7BIZFY1+qMcUKGWb3lBRFUXIktSJJg76uDoucHYiIiWfi\nzmtIKbN7SoqSKWFhYaxYseKt+i5evJjIyMh3Npf169czcuTIdzZeRh07doy2bdu+cb83ne+dO3f4\n9ddf3/g4GRUaGkqLFi2oWLEiLVq04OnTp6m2u3fvHi1btqRq1apUq1aNO3fuZMl8VCBJR8XihXBp\nXYUjNx+z9eL97J6OomRKTgokeV16gSQ+Pj7T48+dOxcnJydu3bqFk5MTc+fOTbVdnz59GD9+PN7e\n3ly4cIFixYpl+tipUZe2XqNvg7L85R3EzH1eNCxflDJFC2T3lJQ84Nu9N/AKfP5Ox6xmWZhp7aqn\n+bmLiwu+vr44ODjQokULihUrxvbt24mJiaFTp058++23RERE4OzsTEBAAAkJCUyZMoWgoCACAwNx\ndHTEzMwMd3f3VMc/ePAgkyZNIiEhATMzM44cOUJoaCgDBgzAz88PIyMjVq9ejZ2d3Sv9XF1d+fbb\nb9HV1cXY2JgTJ05w584devfuTUREBAA//vgjDRs25NixY0ybNo3ixYvj4eFB586dsbW1ZcmSJURF\nRbFr1y7Kly9Pv379MDQ05MaNGwQFBbFw4cIUK5GIiAhGjRrFtWvXiI+PZ/r06XTo0CHN7+/+/ft8\n8skn+Pv707NnT6ZNm8aUKVMwMzNjzJgxAEyePJnixYvz66+/4u3tjYODA3379qVIkSLs37+f6Oho\nIiIiOHr0KPPnz0/x/QNs3ryZpUuXEhsbS7169VixYgW6urqvzGX37t0cO3YMgL59+/Lxxx8zb968\nV9p4eXkRHx9PixYtAChYsGCa55ZZKpC8ho6OYEE3e1otOsGX2zzYPrQBerpqIafkPnPnzuX69et4\neHhw+PBhduzYwYULF5BS0r59e06cOEFwcDCWlpbs378fgGfPnmFsbMzChQtxd3fHzMws1bGDg4MZ\nPHgwJ06cwNramtDQUACmTZtGjRo12LVrF0ePHqVPnz54eHi80nfGjBkcOnQIKysrwsLCAChWrBhu\nbm4YGhpy69YtevTokZR49erVq3h7e2Nqakq5cuUYNGgQFy5cYMmSJSxbtozFixcD2lXB8ePH8fX1\nxdHRkdu3b79y3FmzZtGsWTPWrl1LWFgYdevWpXnz5hQokPovixcuXOD69esYGRlRp04d2rRpw8CB\nA+ncuTNjxoxBo9GwdetWLly4gJ2dHQsWLGDfvn2A9tLY2bNn8fT0xNTUlMOHD3Pr1q0U37+5uTnb\ntm3j9OnT6OvrM2LECLZs2UKfPn0YNGgQw4YNo3bt2gQFBWFhYQGAhYUFjx8/TjFfHx8fTExM6Ny5\nM/7+/jRv3py5c+emCErvggokGWBhnJ+ZHW0Ys9WDn0748YVjheyekpLLpbdyeB8OHz7M4cOHqVGj\nBgAvXrzg1q1bNG7cmHHjxjFhwgTatm1L48aNMzTeuXPnaNKkSdIDbaampgCcOnWKnTt3AtCsWTNC\nQkJ49uzZK30bNWpEv379cHZ2pnPnzoA2C8DIkSPx8PBAV1cXHx+fpPZ16tRJ+ke0fPnytGzZEgBb\nW9tXVkvOzs7o6OhQsWJFypUrx82bN1N8B3v27GHBggWA9jmfe/fuUbVq1VTPsUWLFhQtWhSAzp07\nc+rUKcaOHUvRokX5+++/CQoKokaNGkltUuv/8ntJ6/v39PTk8uXL1KlTB4CoqKiky1Fr1qxJddy0\nxMfHc/LkSf7++29Kly5N9+7dWb9+PQMHDnyjcTJCBZIM6uBghZtXEIvcfGhayRwbK+PsnpKivDUp\nJRMnTmTo0KEpPrt8+TIHDhxg4sSJtGzZkqlTp2ZovNSeRUjtJpX/tlu1ahXnz59n//79ODg44OHh\nwbJlyyhevDhXr15Fo9FgaPjvXZMGBgZJf9bR0Un6WUdH55X9h/8e578/SynZuXMnlStXfu35pTfe\noEGDWL9+PY8ePWLAgAFp9k++0knr+1+2bBl9+/Zlzpw56c6lePHiPHz4EAsLCx4+fJjq3kfJkiWp\nUaMG5cqVA6Bjx46cO3cuSwKJukbzBr7raEPRgvn4cpsH0XEJ2T0dRXkjhQoVIjw8HIBWrVqxdu1a\nXrx4AcCDBw94/PgxgYGBGBkZ0atXL8aNG8eVK1dS9E1NgwYNOH78OP7+/gBJl7aaNGnCli1bAO0d\nU2ZmZhQuXPiVvr6+vtSrV48ZM2ZgZmbG/fv3efbsGRYWFujo6LBp0yYSEt78/2+urq5oNBp8fX3x\n8/NLETBatWrFsmXLkoLd33//ne54bm5uhIaGJu3FNGrUCIBOnTpx8OBBLl68SKtWrYDXf19pff9O\nTk7s2LEj6VJVaGgod++mLAnSvn17NmzYAMCGDRtS3dupU6cOT58+JTg4GICjR49SrVq1dM/xbakV\nyRswMcrH/K729Fl7ge8P/sPUdlnzH0VRskLRokVp1KgRNjY2tG7dmp49e9KgQQNAuxG7efNmbt++\nzfjx49HR0UFfX5+VK1cCMGTIEFq3bo2FhUWqm+3m5uasXr2azp07o9FokvY4pk+fTv/+/bGzs8PI\nyCjpH7/kxo8fz61bt5BS4uTkhL29PSNGjKBLly64urri6OiY5r5FeipXrkzTpk0JCgpi1apVr6xq\nAKZMmcLYsWOxs7NDSknZsmWT9jRS89FHH9G7d29u375Nz549qV27NgD58uXD0dERExOTpP0HOzs7\n9PT0sLe3p1+/fhQpUuSVsVq2bIm3t3eK779atWp89913tGzZEo1Gg76+PsuXL6dMmTKv7JG4uLjg\n7OzML7/8QunSpXF1dQXg0qVLrFq1ijVr1qCrq8uCBQtwcnJCSkmtWrUYPHjwG3+PGSE+hOcjateu\nLd9lhcRpu6+z4exdtgyqR6MKqW8+Ksp/eXt7p3n9XXm3+vXrR9u2benatWuWH0uj0VCzZk1cXV2p\nWLFilh8vq6T291MIcVlKWft1fdWlrbfg0roq5cwLMM71Ks+i4rJ7OoqiZBMvLy8qVKiAk5NTrg4i\nmaUubb2F/Pl0WeTsQOeVZ5i2+zqLP6uR3VNSlPemXr16xMTEvPLepk2bsLW1zaYZpbR+/fq36nfo\n0CEmTJjwynvW1tb88ccfqbavVq0afn5+b3WsvCRLA4kQ4hNgCaALrJFSzv3P5/2A+cCDxLd+lFKu\nSfxsHtAm8f2ZUsptie9bA1sBU+AK0FtKGZuV55Ea+1ImjG5WkUV/+dC8WnHa2lm+7ykoSrY4f/58\ndk8hy7Rq1Sppw1zJuCy7tCWE0AWWA62BakAPIURqu9PbpJQOia+XQaQNUBNwAOoB44UQL2/1mAcs\nklJWBJ4C7/5etgz6wrE89qVMmPzHdR49U1XvFEX5MGXlHkld4LaU0i9xxbAVSDv/wKuqAcellPFS\nygjgKvCJ0N643QzYkdhuA9DxHc87w/R0dVjkbE9MfAJf7/RUiR0VRfkgZWUgsQKSZzoMSHzvv7oI\nITyFEDuEEKUS37sKtBZCGAkhzABHoBRQFAiTUr586iitMRFCDBFCXBJCXHp5H3VWKGdekMltqnHC\nJ5jN51Le760oipLXZWUgSa3k1n9/Zd8LlJVS2gF/oV1hIKU8DBwAzgC/AWeB+AyOSeIYq6WUtaWU\ntc3Nzd/qBKJ9fIgNCHhtu171StO0kjmzDnjjG/zirY6lKIqSW2VlIAlAu4p4qSQQmLyBlDJESvny\n9o+fgVrJPpuVuG/SAm0AuQU8AUyEEHppjfmuyLg4AoYN5/6wYSSk84QqaFMlfN/VDkN9Xb7a5kFc\ngiYrpqQomZKT0sireiSZk5F6JO7u7jg4OCS9DA0N2bVrV5bMJysDyUWgohDCWgiRD/gM2JO8gRDC\nItmP7QHvxPd1hRBFE/9sB9gBh6V2E8IdePmUUV9gd1ZMXujrYzF7FrF37vLgq/8hX1NDoHhhQ2Z3\nsuVqwDOWu99Ot62iZIecFEjyupxQj8TR0REPDw88PDw4evQoRkZGSQku37Usu/1XShkvhBgJHEJ7\n++9aKeUNIcQM4JKUcg8wWgjRHu1lq1CgX2J3feBkYlK050CvZPsiE4CtQojvgL+BX7LqHArUr0+J\naVN5NGUqQXPnUeKbyem2/9TWgk41rFh29DYfVy6GQymTrJqaktv96QKPrr3bMUvYQuvUCxyBqkfy\nodUjSW7Hjh20bt0aIyOjNNtkipQyz79q1aolM+PRnLnSq3IVGbJly2vbhkXGygaz/5KO891lZEx8\npo6r5C1eXl7//nBggpRrP323rwMT0j2+v7+/rF69upRSykOHDsnBgwdLjUYjExISZJs2beTx48fl\njh075KBBg5L6hIWFSSmlLFOmjAwODk5z7MePH8uSJUtKPz8/KaWUISEhUkopR44cKadPny6llPLI\nkSPS3t5eSinlunXr5BdffCGllNLGxkYGBARIKaV8+vSplFLKiIgIGRUVJaWU0sfHR778/7C7u7s0\nNjaWgYGBMjo6WlpaWsqpU6dKKaVcvHixHDNmjJRSyr59+8pWrVrJhIQE6ePjI62srGRUVJR0d3eX\nbdq0kVJKOXHiRLlp06ak41asWFG+ePEi1fNbt26dLFGihHzy5ImMjIyU1atXlxcvXpT+/v6yRo0a\nUkopExISZLly5eSTJ09eOc7L/lZWVknfS1rfv5eXl2zbtq2MjY2VUko5fPhwuWHDBimllAMHDpQX\nL16UUkppbGz8yvxMTEzS/G8jpZSOjo5y79696bZ55e9nIrS/9L/231j1ZHt6wu6Bbj6KjR9H7J07\nBM2aTb7SZSj4UaM0uxjn12eBsz09fz7PnD+9mdHB5j1OWMk10lk5vA+qHkner0fy0sOHD7l27VqW\nPmipAklaEuJgfVsoYI7otx/LBQu4+/nnPBg7lrLbtmJQvnyaXRuWN2PgR9b8csofp6rFaVrp7e4a\nU5SsIlU9kjxfj+Sl7du306lTJ/T19dMdMzNU0sa06OpDq1nw4BLsHY1uASNKrViOMDTk/rDhxKdy\nl0Ry41tVpmKxgox3vcrTiPeewUVRUlD1SD6seiQv/fbbb/To0SPdc8ssFUjSU7UdOH4Dntvg9GL0\nrawotfxH4oOCCBg1Ck1s2gHCUF+XRd0deBoZyze7rqun3pVsl7weiZubW1I9EltbW7p27Up4eDjX\nrl2jbt26ODg4MGvWLL755hvg33okjo6OqY6dvB6Jvb093bt3B2D69OlcunQJOzs7XFxc0qxHYmtr\ni42NDU2aNEmqR7Jhwwbq16+Pj49PpuqRtG7dOs16JHFxcdjZ2WFjY8OUKVPSHe9lPRIHBwe6dOmS\noh6Js7NzqvVIFi1alGKsli1bpvr9J69HYmdnR4sWLXj48CGgXfm8LIfh4uKCm5sbFStWxM3NDRcX\nF0Bbj2TQoEFJx7lz5w7379+nadOmb/z9vZGMbKTk9lemNts1Gild+0s5zVjKmweklFKG7dsnvSpX\nkQ8muEiNRpNu9x+P3pJlJuyTf1wJePs5KHlCapuZStbo27evdHV1fS/HSkhIkPb29tLHx+e9HC+r\nZGazXa1IXkcI6LAcLB1g5yAI8sK4TRvMRo7k2a5dhLxmA2xY0/LUKlOEKbuvExgW9Z4mrSjK+6Dq\nkWipCokZ9TwQVjuCXj4YfAxpZErg/8bx/M8/sVq6hMItWqTZ9W5IBK2XnMShlAmbB9ZDRye1TC9K\nXpdXKiTmhnokb+tN65HkJZmpkKgCyZsIuAzrWkPJOtD7DzTxGu727UuMzy3KbN5E/urV0+y69cI9\nXH6/xtS21RjwkXXm56LkOnklkCh5kyq1+76UrKW9zHX3FBwYh46BAaV+/BHdIiYEDB9BXNDjNLt2\nr1OK5lWLMffgTW4FpZ+7S1EUJTdRgeRN2XWDj76CKxvgwmr0zM0ptXIlmhcvCBgxAk1U6vsgQgjm\ndLajkIEeY7d5EBuvEjsqipI3qEDyNppNgcpt4OBE8D2KYeXKWP6wgGgvLwInuCA1qQcJ80IGzO5s\ny43A5yw9cus9T1pRFCVrqEDyNnR0oPNPYF4FXPvBk9sUcnSk2NdfE374MMFLl6bZtVX1EjjXLsmK\nY7e5fDf0/c1Z+eDlpOy/Ko185mQkjTzA119/TfXq1alatSqjR4/OsufZVCBJx7Xga4RGp/GPvUEh\n6PEb6OjBb90hKgzTfn0x6daNkFU/8Wx32tntp7SthqVJfr7cdpWImMynlFaUjMhJgSSvywlp5M+c\nOcPp06fx9PTk+vXrXLx4kePHj2f62KlRgSQNcZo4xh0fR+fdnTl2/1jqjYqUAedN8PQu7BiA0CRQ\nYuoUjOrX5+E3U4i8fDnVboUM9Vno7MD9p5F8t987605CUZJJnkZ+/PjxzJ8/nzp16mBnZ8e0adMA\nbWr1Nm3aYG9vj42NDdu2bWPp0qVJaeTTerIdtGnka9asib29PU5OToD2N+eOHTtiZ2dH/fr18fT0\nTNHP1dUVGxsb7O3tadKkCaD9h7hx48bUrFmTmjVrcubMGUC7omjatCnOzs5UqlQJFxcXtmzZQt26\ndbG1tcXX1xeAfv36MWzYMBo3bkylSpXYt29fiuNGREQwYMAA6tSpQ40aNdidzi9/8G8a+cqVKyel\nfJ8yZQpLlixJajN58mSWLl2Ki4sLJ0+exMHBgUWLFrF+/Xq6detGu3btkpJMpvb9gzaN/MvsAkOH\nDk01Pczu3bvp27cvoE0jn1rBKiEE0dHRxMbGEhMTQ1xcHMWLF0/3HN+WStqYBn0dfZY2W8qkU5MY\ndXQUnSt2Znzt8RTMV/DVhmUbQduFsGcUuE1BfDKHkosXcaf7ZwSMHEXZ7dvIV6pUivHrWpsypEk5\nfjruR/OqxXCqmjX/gZWcad6FedwMvfn6hm+gimkVJtSdkObnc+fO5fr163h4eHD48GF27NjBhQsX\nkFLSvn17Tpw4QXBwMJaWluzfvx+AZ8+eYWxszMKFC3F3d8fMzCzVsYODgxk8eDAnTpzA2to6KdfW\ntGnTqFGjBrt27eLo0aP06dMHDw+PV/rOmDGDQ4cOYWVlRVhYGADFihXDzc0NQ0NDbt26RY8ePZLS\ng1y9ehVvb29MTU0pV64cgwYN4sKFCyxZsoRly5axePFiQBuMjh8/jq+vL46Ojty+/WrBuVmzZtGs\nWTPWrl1LWFgYdevWpXnz5mmmY7lw4QLXr1/HyMiIOnXq0KZNGwYOHEjnzp0ZM2YMGo2GrVu3cuHC\nBezs7FiwYEFSAFu/fj1nz57F09MTU1NTDh8+zK1bt1J8/+bm5mzbto3Tp0+jr6/PiBEj2LJlC336\n9GHQoEEMGzaM2rVrExQUlJQB2cLCIik3V3INGjTA0dERCwsLpJSMHDkyy24/VyuSdFQ2rczWNlsZ\nbDuYXbd30WVPFy4+upiyYc0+UG84nFsBVzaia2JCyVUrkRoN94cPT7NU71ctKlGlRCEm7PQk5EVM\nqm0UJSskT2Nes2ZNbt68ya1bt7C1teWvv/5iwoQJnDx5EmNj4wyNl14a+d69ewOvTyP/888/J/32\nHRcXx+DBg7G1taVbt254eXkltX+ZRt7AwCBFGvk7d+4ktctIGvm5c+fi4ODAxx9/nJRGPi0v08jn\nz58/KY182bJlk9LIv/w+3zSNfPLv/8iRI0lp5B0cHDhy5Ah+fn6ANo38y/xeGXH79m28vb0JCAjg\nwYMHHD16lBMnTmS4/5tQK5LX0NfVZ3TN0TQp2YTJpyYz8NBAelfrzeiaozHQ/TedNS2/gyf/wL6v\noGgFDKwbUnLJEu4NGsSDr/5HqZUrEHqvft0Geros/syB9stOM/H3a/zUu1aqqbiVvCe9lcP7IFUa\n+TyfRv6PP/6gfv36FCyovYrSunXrpID/rqkVSQY5FHPAtZ0rzpWd2ei1ke57u3Mj5Ma/DXT1oOs6\n7b7Jtl7w9C4F6tejxLSpRJw8SdDc1MtgVilRmHGtKnHYK4gdlwPe09koHyKVRv7DSiNfunRpjh8/\nTnx8PHFxcRw/flxd2soJjPSN+Kb+N6xqvorw2HB67e/FqquriNck/haU3wR6bANNPPzWA2JeUKRb\nN0z79+fp5s2EpnEXx8CPylHP2pRv93pxP1TdGaNkDZVG/sNKI9+1a1fKly+Pra0t9vb22Nvb065d\nuzf+HjMkIymCc/srszXbUxMWHSa/Pv61tFlvI3vs6yH9wvz+/fD2ESmnF5Hy1x5SJiRITXy8vDd0\nmPSqVl2GnzyV6nj3QyNk9akHZbeVZ2R8Qvqp6ZXcSaWRf39UGvk3p9LIZwNjA2PmNZnHgqYLuBd+\nj257u7HFewsaqYHyzeCTOfDPfnD/DqGri+WCBRhUqMCDsWOJSbxFMbmSRYyY3r46F+6EsuakXzac\nkaIob0qlkddS2X/fgeDIYKadmcbJByepV6IeMxvNxKJACdg7RpuTq/MasOtGXGAg/s7d0cmfn7Lb\nt6FXpMgr40gpGb75CkdvPmb3yEZUtSicxhGV3CivZP9VaeTzJpVG/jWyOpBA4h0gt3Yy/+J8dIQO\nE+tNpF3pVojNnSDgEvT/E0rWIurqVe727oOhnS2l165FJ1++V8YJjYil5aITmBXMx+6RjTDQ083S\neSvvT14JJErepNLI5wBCCLpW6sqO9juoVKQSk09N5stTEwjtsAwKFYetPeF5IPnt7bGcO4eoS5d5\nNHVaitsjTQvk4/uuttx8FM7Cwz7ZdDaKoigZpwLJO1aqUCnWtlrL/2r9jxMBJ+jkNoCjzcZB7Att\nMImLovCnn6ZbqrdZleL0rFea1Sf9OO8Xkg1noSiKknEqkGQBXR1d+tn0Y1vbbRQzKsaYvxfwjX0L\nwh9ehd1fgJSYfTGCwm3aEPzDQp67uaUYY/KnVSltasRX268SHh2XDWehKIqSMSqQZKGKRSry66e/\nMth2MHtD/qZLhcpcuL0PTi5ACIHFrO8wtLcj8OsJRN248UrfAgZ6LHR24OGzKL7d65XGERRFUbKf\nCiRZ7GWKlY2tN5IvvxkDLYozz+NHom/8jo6hYbqlemuVKcIXjhXYcTmAg9cfZdMZKHlFTkojr+qR\nZE5G65FMmDABGxubpEzOWUUFkvfE3tye7e2206OSM5uNC+N8djI3/tn92lK9o50qYmNVmEl/XONx\neHQ2zV7JC3JSIMnrckI9kv3793PlyhU8PDw4f/488+fP5/nz55k+dmpUIHmPjPSNmNRgCj81mkuE\nji6fn53MygsL0K1YLs1Svfq6OixydiAiJp6JO69lWYUzJe9T9UheldfrkXh5edG0aVP09PQoUKAA\n9vb2HDx4MN1zfGsZefw9t7+yIkVKZoXdOSldlleQNuttZPc9ztL3qa98sm6d9KpcRQYtXJSi/S8n\n/WSZCfvkr+fvZsNslXcheQqKh7NmyTu9er/T18NZs9I9vr+/v6xevbqUUspDhw7JwYMHS41GIxMS\nEmSbNm3k8ePH5Y4dO+SgQYOS+oSFhUkppSxTpowMDg5Oc+zHjx/LkiVLSj8/baqgkJAQKaWUI0eO\nlNOnT5dSSnnkyBFpb28vpZRy3bp18osvvpBSSmljYyMDAgKklFI+ffpUSillRESEjIqKklJK6ePj\nI1/+f9jd3V0aGxvLwMBAGR0dLS0tLeXUqVOllFIuXrxYjhkzRkqpTZHSqlUrmZCQIH18fKSVlZWM\nioqS7u7usk2bNlJKKSdOnCg3bdqUdNyKFSvKFy9epHp+69atkyVKlJBPnjyRkZGRsnr16vLixYvS\n399f1qhRQ0qpTZVSrlw5+eTJk1eO87K/lZVV0veS1vfv5eUl27ZtK2NjY6WUUg4fPlxu2LBBSinl\nwIED5cWLF6WUUhobG78yPxMTkxRzPnTokGzYsKGMiIiQwcHB0traWi5YsCCt/4SZSpGi0shnE+My\nHzGnyTwcD4xgpvDBeZ8zY+qMxqlbV0J++gmDctYYJ8vo2a9hWY7cDGLmPi8ali9KmaJvnsROUV5K\nXg8D4MWLF9y6dYvGjRszbtw4JkyYQNu2bWncuHGGxkuvHsnOnTuB19cjcXZ2pnPnzoC2HsnIkSPx\n8PBAV1cXH59/n6l6WY8ESFGPxN3dPaldRuqR7NmzhwULFgAk1SNJ66HRl/VIgKR6JGPHjk2qRxIU\nFPRW9Ujg3+/f09MzqR4JQFRUVFKK+DWpPCqQnpYtW3Lx4kUaNmyIubk5DRo0QE8va/7JV4EkO9l0\npmXwTWqems/0qo34/tJ8jjeoxdd3avDwmynolyyJUa1aAOjoCBZ0s6fVohN8uc2D7UMboKerrkzm\nViUmTcrW40up6pFImbfrkYD2UtvkyZMB6NmzZ5blA1P/EmW3pi6YVWrLshunmVH+M6499WbYx77E\nFDMmYOQoYu/fT2pqYZyfmR1tuHIvjJ9OqMSOyptR9Ug+rHokCQkJhIRoH2j29PTE09MzafX2rqkV\nSXbT0YFOqxC/tKLTyVXU6flgfdH4AAAgAElEQVQr33j/wri2l/h+k+DO0CGU37Yd3UKFAGhvb4mb\nVxCL3HxoWskcG6uMlUJVlOT1SFq3bp1UDwOgYMGCbN68mdu3bzN+/Hh0dHTQ19dn5cqVwL/1SCws\nLF65fPRS8nokGo0mqeb69OnT6d+/P3Z2dhgZGaVZj+TWrVtIKXFyckqqR9KlSxdcXV1xdHTMVD2S\noKCgNOuRjB07Fjs7O6SUlC1bNtVN+Zde1iO5ffs2PXv2TFGPxMTEJNV6JP369aPIfxK0tmzZEm9v\n7xTff/J6JBqNBn19fZYvX06ZMmVeqdnu4uKCs7Mzv/zyC6VLl8bV1RXQ1iNZtWoVa9asIS4uLunS\nZOHChdm8eXOWXdpSSRtzirD78LMjGBRCM9CNTXf24/b7Ilx+iyGuRlUcNmxPKtUbFhlLq8UnKGyo\nz95RH2GorxI75gYqaeP7069fP9q2bUvXrl2z/FgajYaaNWvi6uqaq1PJq6SNeYFJKei+BZ4FoLNj\nAH2r9GT6yB3s62yF4WVvdo3uSHisdqlsYpSP+V3tufX4Bd8f/CebJ64oHy5Vj0QrSwOJEOITIcQ/\nQojbQgiXVD7vJ4QIFkJ4JL4GJfvseyHEDSGEtxBiqUjc2RJC9BBCXBNCeAohDgohzLLyHN6r0vWg\n7WLwPw6HJlGhSAX+N+NP7rZxoMpRXxZObsm5h+cAaFLJnL4NyrD2tD+nbz/J5okrH5J69erh4ODw\nyuvatWvZPa1XrF+//q1WI4cOHUpxbp06dUqzfbVq1fDz8+OHH37IzHRzvSy7tCWE0AV8gBZAAHAR\n6CGl9ErWph9QW0o58j99GwLzgSaJb50CJib+byBQTUr5RAjxPRAppZye3lxyxaWt5A5/A2eWQdtF\nUHsAMiEBryF9kWcvM6ebDrZtejOm5hjQ5KPNspNExSZwcGwTjPPrZ/fMlXSoS1tKTpZTL23VBW5L\nKf2klLHAViDlrQWpk4AhkA8wAPSBIEAkvgokrlAKow0seUvzb6FiSzgwHvxPInR1qbr0JwwrVGTC\nHl2OndyM815nbj/zYpGzA4/DY5i2+3p2z1rJgA9hT1LJfTL79zIrA4kVcD/ZzwGJ7/1Xl8TLVDuE\nEKUApJRnAXfgYeLrkJTSW0oZBwwHrpG4MgF+ycJzyB46utBlDZiWh+29IdQfnQIFKLNqFYYFjFmw\n3xSdZy/o/WdvToVsYaSjNbs8Atnnmfdial5iaGhISEiICiZKjiKlJCQkJMVdbW8iKy9tdQNaSSkH\nJf7cG6grpRyVrE1R4IWUMkYIMQxwllI2E0JUAJYA3RObugETgLPAQWAI4AcsAx5JKb9L5fhDEttR\nunTpWqndi53jhfjCGicoWBwGuoFh4aRSvfo21Vg/uAy77u2nqmlVwu93IzDYmENjm1DC+O3/QihZ\nJy4ujoCAAKKjVfJNJWcxNDSkZMmS6Ou/enk822u2CyEaANOllK0Sf54IIKVM9ZHNxD2VUCmlsRBi\nPGAopZyZ+NlUIBrtKmWulNIp8f0mgIuU8tP05pLr9kiS8zsOmzpBxRbw2a+go8vzAwd48NX/MO7Y\nEa9hzZhxbiYv4iKIftySWkU6sLF/vVSfMlYURXkTOWGP5CJQUQhhLYTIB3wG7EneQAhhkezH9oB3\n4p/vAU2FEHpCCH2gaeJnD4BqQgjzxHYtkvXJm8o1hU+/B5+DcESbcbTwp59iNkpbqtfB7Q6/d/id\nRlYN0TXbx6Xo2Sw/eSGbJ60oyocky55sl1LGCyFGAocAXWCtlPKGEGIG2oySe4DRQoj2QDwQCvRL\n7L4DaIZ2L0QCB6WUewGEEN8CJ4QQccDdZH3yrjqD4LE3nF4C5lXBoQdmI0YQ6+dP8A8LsSpblqXN\nl7Lr9i6mn57NKt8v0C/4P4bU+EytTBRFyXLqyfbcIiFOe4nr/nnodwBK1UETE8O9Pn2J9vGhzOZN\n5K9enasP/ei15ysw9KWJVVO+bTQds/x551EbRVHen5xwaUt5l3T1wXkjFLaCrT21T8AbGFByefJS\nvUHYW5Rjdv3lRD9qy+kHZ+i0uxNud92ye/aKouRhKpDkJkam0HMbxEfDbz0gNgI9M7NkpXq/QBMV\nRTt7K9qW7c4L/1EUyVecr459xcSTE3kemzVlNhVF+bCpQJLbmFeGLr/Ao2uwazhoNBhWrozlwh+I\n9vZOKtU7vX11ihmUJtxvOINshvKn/5903t2ZM4FnsvsMFEXJY1QgyY0qtYSWM8FrN5z4HoBCH39M\nsa/HE374MMFLlmKcX58F3ezxfxJNSMDHbP50M0b6Rgx1G8rs87OJio/K5pNQFCWvUIEkt2owEhw+\nh2Nz4MYfAJj27YuJszMhP/3Es927aVjBjIEfWbPx7F1CQouzve12elXtxW83f8N5rzOewZ7ZfBKK\nouQFKpDkVkJokzqWqgd/DIeHVxFCUGLKNxjVr8/Db6YQefky41tVpmKxgox3vUpUjA4T6k5gTcs1\nxCTE0PvP3iz7exlxCXHZfTaKouRiKpDkZnoG0H0zGBXVbr6HByH09Sm5ZDH6VlYEjByFzqNAFnV3\n4GlkLN/suo6UknoW9djZfiftyrVjtedqPj/wObee3srus1EUJZdSgSS3K1gMevwGUU9h2+cQF42u\nsTGlVq1EajTcHz6cqoV1GNu8EvuvPWS3hzaxY6F8hfjuo+9Y7LiYoMgguu/rzvrr60nQvHltbEVR\nPmwqkOQFFnbQ6ScIuAj7xoKU5CtblpJLlxJ75y4PvvyKoY3KUKtMEabsvk5g2L8b7U6lnfi9/e80\ntmrMD5d/YMChAQS+UFmEFUXJOBVI8opq7cFxMlz9Dc4sBaBAvbqUmDaViFOnePL99yx0tidBIxnn\nehWN5t+MBkXzF2Wx42JmfTQLn6c+9DrQS13qUhQlw1QgyUuajIfqncFtGvxzEIAi3bph2r8/Tzdv\nptCh3UxtW40zviGsO3Pnla5CCNqXb8/G1hsB6HuwLx6PPd73GSiKkgupQJKXCAEdloOFPexMTPQI\nFBv3Pwo6OhI0azZtY+/RvGox5h28ya2g8BRDVCxSkY2tN1LEoAhD3IZw6sGp930WiqLkMq8NJEKI\n8kIIg8Q/fyyEGC2EMMn6qSlvJZ+Rtm5JPiP47TOICEHo6mK1YD4GFSvyYOyXzHQoQEEDPcZu8yA2\nXpNiiJKFSrKh9QZKFyrNqCOj+NP/z2w4EUVRcouMrEh2AgmJVQt/AayBX7N0VkrmGFtpg8nzh+Da\nFxLi0ClQgFIrliMMDYkYN5Z5zUtzI/A5S4+kvhdilt+MdZ+sw87cjgknJrD15tb3fBKKouQWGQkk\nGillPNAJWCyl/BKweE0fJbuVrA0dfoQ7J+HAeJASfUtLSi3/kfjHj6m4fCbd7Yuz4thtLt8NTXWI\nQvkK8VOLn2hasimzzs9i5dWVqt64oigpZCSQxAkhegB9gX2J7+mn017JKeyc4aMv4fI6uLgGgPz2\n9ljOmU3UpcuMuOKKpbEhX267SkRMfKpDGOoZstBxIe3Lt2eFxwrmXZyHRqa8HKYoyocrI4GkP9AA\nmCWl9BdCWAObs3ZayjvTbCpUag1/TgC/Y8C/pXoj9+7hR3GD+08j+W5/2hWL9XX0mdloJr2q9mKL\n9xYmnZpEnEalVVEUReu1gURK6SWlHC2l/E0IUQQoJKWc+x7mprwLOjrQ5Wdt+vntfSHEFwCzESMo\n3KYNButWMs3kMb9duMcR76C0hxE6fF3na0bXGM1+v/2MOTpGZRBWFAXI2F1bx4QQhYUQpsBVYJ0Q\nYmHWT015ZwwKadOoCB34tTtEhSGEwGL2LPLb21N/6zKa6z5lwk5PQl7EpDmMEILBdoOZUn8Kpx6c\nYqjbUFUsS1GUDF3aMpZSPgc6A+uklLWA5lk7LeWdK1JWm+DxqT/sHAiahFdK9f7v+Gr0Qp8w8fdr\nr91Qd67szPdNv+fak2v0P9if4Mjg93MOiqLkSBkJJHpCCAvAmX8325XcqGwjaPMD3P4L3KYCJJbq\nXYVOZARLb/zKcc/7fLvXi6Dn0ekO9UnZT1jutJz74ffp82cf7offfx9noChKDpSRQDIDOAT4Sikv\nCiHKASoRU25Vqx/UGwZnf4QrmwAwrFwJy4U/UOi+H4v9d7HpjB8fzTvKeNer+KTy9PtLDS0bsqbl\nGsLjwunzZx/+Cf3nPZ2Eoig5ifgQnguoXbu2vHTpUnZPI+dIiIctXeHOKei3D0rXByB0wwaC5sxF\nt8UnbGv0GVtuhBIdp8GxsjlDmpSnfjlThBAphvMN82WI2xCi4qNY7rScGsVqvO8zUhQlCwghLksp\na7+uXUY220sKIf4QQjwWQgQJIXYKIUq+m2kq2UJXD7qtgyJlYOvnEHYPgCJ9+mA+dgwJR934bOUE\n3Jvm58vmlfAMeEaPn8/RYflp9nkGEp/w6nMk5U3Ks6n1JkwNTRlyeAgnAk5kx1kpipJNMnJpax2w\nB7AErIC9ie8puVn+ItBjKyTEaasrxrxACIHZsGGU3boVnUIFCRs5HOdTWzg5uj6zOtkQHh3PyF//\nxvGHY6w/7U9k7L8PMVoWtGTDJxuwNrZmzNEx7PNT22mK8qF47aUtIYSHlNLhde/lZOrSVjpuH9Fe\n5qr8KThv0j53AmhiYgheupTQtevQt7LCcs5sDGrVxs0riNUnfLlyLwwTI3161y9DnwZlMS9kAMCL\n2BeMOjqKS0GXcKnrwudVP8/Os1MUJRPe2aUt4IkQopcQQjfx1QsIyfwUlRyhghO0mg0398Gx2Ulv\n6xgYUHz8eMps2Qw6Otzt05cn8+bRsoIJv49oxM7hDahb1pQf3W/TaN5RJv5+Dd/gFxTMV5BVLVbh\nWMqRuRfmssJjhcrPpSh5XEZWJKWBH9GmSZHAGWC0lPJe1k/v3VArkteQEvaOhisbocsvYNv1lY81\nkZE8XvADT3/9lXzW1ljOm0t+OzsAfINfsOakPzuvBBCXoKF51eIMbVIOh9KFmX5mOrt9d/NZ5c+Y\nWG8iOkKVv1GU3CSjK5K3umtLCDFWSrn4rWaWDVQgyYD4WNjYAQKvQJ/dSXdyJRdx9iyBkyYTHxRE\n0cGDMftiBDr58gEQHB7DprN32HjuLmGRcdQsbcLgxuW4Hr2ZjV4baW3dmlmNZqGvq/J9KkpukdWB\n5J6UsvRbzSwbqECSQRFPYE1zeBYALb+DekO1VReTSQgPJ2juXJ7t/B2DypWxnDcXwypVkj6PjI3H\n9VIAa075cT80irJmRthUvcLxJ+tpZNWIhU0XYqRv9L7PTFGUt5DVgeS+lLLUW80sG6hA8gYiQ2HX\nCPD5E6q2g/Y/Qv6UBTHD3d15OHUqCWHPMP9iBEUHDULo6SV9Hp+g4eCNR6w+4YdnwDNMil1GU3Qn\n1YvasqrFcowNjN/nWSmK8hbUiiQZFUjekJTaJ9/dpoFJKei2ASxT3qQX//QpQTO/4/mBAxja2mI5\ndw4G5cv/ZyjJef9QVp/w48SDIxhabcVY15JlzVZQ06rsezohRVHeRqYDiRAiHO3meoqPgPxSSr1U\nPsuRVCB5S/fOw47+EBGsvbOrzqAUl7oAnh88yKPp36KJjMT8yy8x7dMboaubop1PUDhz3fdwIWoh\nMr4gdfNPYEzThjiUSrniURQl+2XpiiS3UYEkEyJC4I+hcNsNqneCdkvBsHCKZvFPnvBw2nReHDlC\n/tq1sJw9m3ylU1+0Hr97mf8dH0VMHETc609tSxuGNC5HsyrF0NFJGagURckeKpAkowJJJmk0cHox\nHP1Om46+23qwsEvRTErJ8z17ePTdLGRCAsW/Ho9J9+6p5ufyC/Nj8OEhhMW8QPfxAB4HW1GhWEEG\nN7amYw0rDPRSrmgURXm/3uUDicqHTkcHGn+lTfAYF6m9s+vSOu1eSjJCCIw7dKDc3j0Y1ajBo+nf\ncn/gIOIePkwxZDmTcmz+dBOWBc3RFF/N8NYx6OvqMGHnNT6a585y99s8i1TlfBUlN1ArEuXNvAiG\nP4aA71Gw7QZtF4NBwRTNpJSEbdtO0PffI3R0KD55MsYdO6RYnYRGhzLirxHcDL3JjIYzMJUN+emE\nLydvPcEony7d65Ri4EfWlCyibhlWlPdNXdpKRgWSd0yjgZM/aFOqmJYH5w1QvHqqTWPv3+fhxElE\nXrpEwWbNsPh2Onrm5q+0iYiLYMzRMZx/dJ4JdSbQq1ovvAKf8/NJP/ZeDUQCbWwtGNKkHDZW6rZh\nRXlf3mUa+XAhxPP/vO4nppYv95q+nwgh/hFC3BZCuKTyeT8hRLAQwiPxNSjZZ98LIW4IIbyFEEtF\n4q+yQoh8QojVQggfIcRNIUSX152D8o7p6EDT8don4GOew89O2iJZqfxSkq9UKUpv3EDxiS5EnD6N\nX7v2PD948JU2BfQLsLz5cpxKOzHv4jx+/PtHqloUYlF3B0587ciARmU5evMxbZed4vM15zj2z2OV\nv0tRcpCM5Nr6FggEfkV76+9nQAngH2C4lPLjNPrpAj5ACyAAuAj0kFJ6JWvTD6gtpRz5n74NgflA\nk8S3TgETpZTHEuejK6X8RgihA5hKKZ+kdw5qRZKFXjzW1oD3PwH2PbSlfPMVSLVpjJ8fgS4Tifb0\npPCnrSk+ZQp6RYokfR6viWfmuZn8fut3nCs5M6neJHR1tJvuz6Li+O3CPdad9ifoeQxVShRicONy\ntLO3JJ+e2upTlKzwLjfbP5FS/iSlDJdSPpdSrgY+lVJuA4qk068ucFtK6SeljAW2Ah0yNHvt8yuG\nQD7AANAHghI/GwDMAZBSal4XRJQsVrAY9N4FTV3g6lb4uRk8vplqU4Ny5Sj76xbMx47ludtf+LVv\nT/hR96TP9XT0mN5gOgNsBrDdZzsuJ12IS9BuuBvn12dY0/Kc/LoZ87vaoZGS/7lepcn37qw+4cvz\naLUxryjZJSOBRCOEcBZC6CS+nJN9lt5yxgq4n+zngMT3/quLEMJTCLFDCFEKQEp5FnAHHia+Dkkp\nvYUQL59cmymEuCKEcBVCFM/AOShZSUcXHCdC7z8gMgR+dgSP31JtKvT0MBs2FGvX7egVNSNgxAgC\nJ00mIVxbG14IwZe1vuSrWl9x8M5BRh4dSWRcZFL/fHo6dKtdikNjm7Cufx2szQow+8BNGs05yuwD\n3jx8FvVeTllRlH9lJJB8DvQGHie+egO9hBD5gZHp9EvtybL/Bp69QFkppR3wF7ABQAhRAagKlEQb\nfJoJIZoAeonvnZZS1gTOAgtSPbgQQ4QQl4QQl4KDgzNwmkqmlXeEoSfBsibsGga7v4DYyFSbGlap\ngvX2bRQdPoxnu3fj174DEWfOJH3e36Y/MxrO4NzDcwx2G8yzmGev9BdC4Fi5GL8Nqc+ekY1oWtmc\nNSf9aDzPna+2e3Dz0fMsPVVFUf6VZXdtCSEaANOllK0Sf54IIKWck0Z7XSBUSmkshBgPGEopZyZ+\nNhWIRrtv8gIoJKXUJK5gDkopU79lKJHaI3nPEuLh2Bw4uQCKVQPnjWBWMc3mUZ6eBLpMJNbPjyI9\ne1Bs3Dh0jLS3+x65e4TxJ8ZTpnAZVjVfRfECaS9A74dG8sspf7ZdvE9UXAJNK5kzpEk5GpYvmupD\nkYqipO9d3rVVMvEOrcdCiCAhxE4hRMkMzOEiUFEIYS2EyId2k37Pf8a2SPZje8A78c/3gKZCCD0h\nhD7QFPCW2qi3F/g4sZ0T4IWSs+jqgdMU6LUTXgTBT03B0zXN5vnt7LD+fSem/frx9Let+HXsROTl\nywA4lXFiVfNVPIx4SJ8/+3D3+d00xyllasT09tU5O7EZ41pW4kbgMz5fc552P55it8cD4hM07/xU\nFUXJ2F1bbmjv2NqU+FYv4HMpZYvXDi7Ep8BiQBdYK6WcJYSYAVySUu4RQsxBG0DigVC0d4HdTFyd\nrEB715ZEu+r4KnHMMolzMQGCgf6vq9aoViTZ6NkD7V1d985CrX7wyTzQN0yzeeSlSwROnERcQACm\n/ftjPmY0OgYG3Ai5wXC34QghWNV8FVWLVn3toaPjEvjj7wf8fNIPv+AIrEzyM/Aja7rXKUUBg1yT\nc1RRss07eyBRCOEhpXR43Xs5mQok2SwhTpun6/RiKG6rfYCxaPk0m2siIghasICw37aSr3x5LOfO\nIb+tLf7P/BnqNpTw2HCWNltKnRJ1MnR4jUZy5OZjVp/w5eKdpxQ21KNX/TL0a1SWYoXSDmqK8qF7\nl4HkL2A98PI2nB5oVwFOmZ3k+6ICSQ7hc0ibSTghHtovBZvO6TZ/ceo0D7/5hvjgYMyGDsFs2DCC\n4kIZ6jaUgPAAFjRdgGNpxzeawpV7T/n5hB8HbzxCX0eHTjWsGNzEmgrFCmXmzBQlT3qXgaQ08CPQ\nAO1lpjPA6NddTspJVCDJQcLuw44BEHAB6gyGVrNAzyDN5gnPnxM0ew7Pdu3CoGpVLOfOJaqMOSP+\nGoF3qDffNvyWDhUy+njSv+48iWDNKT9cLwUQE6+hR93STG5TlYLqkpeiJMnqColjpZSL32pm2UAF\nkhwmIQ7+mq6twmhhr63AaGqdbpfwI0d4OHUaCc+fYz5yJIZ9uvPlyXGce3iOcbXH0bd637eaSsiL\nGFYc82XtaX8sjfPzfVc7GlUwe6uxFCWvUaV2k1GBJIe6uR92Ddeuczv8CNXap9s8/ulTHn07g/CD\nBzG0t8N81kymBqzC7a4bg20HM6rGqLe+zffy3VDGuXri/ySC3vXL4NK6itqQVz54WV2PRN2Ur2Re\nlTbaBxiLloftveFPF4iPTbO5XpEilFy8CKuFPxB35y4BXZyZdNeOrhW68PO1n5lxbgYJmoS3mkqt\nMqYcGN2YgR9Zs/n8XT5ZcoKzviFve2aK8kF520CiUq8q70aRMjDgENQbDudXwrpP4Gnaz4oAFP70\nU6z37qFAgwYEz5lH31V+jCrenR0+Oxh/YjyxCWkHo/Tkz6fLlLbV2D60ATpC0OPnc0zbfZ3I2Pi3\nGk9RPhRpXtoSQoSTesAQQH4pZa5Z96tLW7mE127YPRKEgI6roMqn6TaXUvLs9z8Imj0bpMS3dxMm\nFHGjvmUDljguwUj/7YthRcbG8/3Bf1h/5g6lTY1Y0M2eutambz2eouRGqrBVMiqQ5CKhfuDaDx5e\nhQYjofl00NVPt0tcYCCBkycTefYcETUrMb7RHSzKVmeF0wpMDE3S7fs65/xC+HqHJ/efRtK/oTXj\nW1Umfz5VT175MKhAkowKJLlMXDQcngwX10DJutBtHRinn5VHajQ83bqVx/MXkKALPzVL4G6DMvzU\ncjUlCpTI1HQiYuKZd/AmG8/exdqsAPO72lG7rFqdKHmfCiTJqECSS13fCXvGaHN3dVoNlVq+tkvs\n3bsETpxE1JUrXK6iz+8dzfmh0xqsjdO/vTgjzvg+4esdnjwIi2JgI2vGtaqMob5anSh5lwokyahA\nkouF+ML2vhB0DRqNhWZTtIElHTIhgdD1G3i8ZDHhevH82q4Qg0evpXrRdJNEZ8iLmHjmHPBmy/l7\nlDMrwAJne2qWTq++m6LkXll9+6+ivB9Fy8MgN23Cx9OLYUNbeB6Ybhehq0vRgQMo9/vvGJeuwNBt\nzzk/tAcXfdzT7ZcRBQ30mNXJls0D6xETr6HryjPM+dOb6Li3u+1YUfICFUiUnE8/P7RbAp1/hoee\nsOojuP3Xa7sZVKhAJded5B82gLpeccT1/IJTO5a+kyl9VNGMg2Mb071OKX467kfbZafwuB/2TsZW\nlNxGBRIl97BzhiHHoGBx2NwVjszUJoBMh9DXp+zY8ZT4dQNxBQ0o+s1Kzo7uTcKLF5meTiFDfeZ0\ntmPDgLpExMTTecVp5h28SUy8Wp0oHxYVSJTcxbwSDDoCNT7XVmDc2AHCH722m5lDXWrvd+dii1IU\ndrvEtU+ciDh37p1MqWklcw592YSutUqy8pgv7ZadwjNArU6UD4cKJEruk88IOiyHjivhwWXtpS6/\nY6/tVqCACT0W72f/140I0TznXr/+PPruOzRRUZmeUmFDfb7vas+6fnV4FhVHpxVn+OHwP8TGq6qM\nSt6nAomSezn0hCHukN8UNnYE9znwmlxb+rr6/K/vT1yY0539tQVPN2/Br0NHIi5ceCdTcqxSjMNj\nm9LRwYplR2/T/sdTXH/w7J2MrSg5lbr9V8n9YiNg31fguRWsm0KXNVCwWLpdpJQs91jOqb0r+fKQ\nAYVDoijcvh3Fv/4aPbN3k0b+iHcQE3+/RmhELCMcKzDSsQL59NTvbkruoZ4jSUYFkg+AlPD3Jjgw\nHgyNocsvYN34td02e21m0Zm59L9cmGYnn6Ob3wjzsWMo8tlnCN3MP2wYFhnLt3u9+OPvB1S1KMwP\n3eypZlk40+MqyvugAkkyKpB8QB5dB9e+2pxdH0+Cxv8DnfRXAWcenGH62emIew+ZfNoc8xsPMaxW\njRLTp5Hfzu6dTOvwjUdM+uM6YZGxjHaqyPCPy6Ovq1YnSs6mAkkyKpB8YGLCYe9YuL4DyjfTPn9S\nIP3LVRFxESy6vIhtN7fSxr8Ivf+KRyf0GSbOzhT7ciy6JplL/gjwNCKWaXtusOdqIDZWhVnQzZ4q\nJdTqRMm5VCBJRgWSD5CUcHmdtliWkSl0XQtlGr6226VHl5h+djqPg+8w6Xp5Kh/xRbdwYYqNG4dx\np46I16xuMuLg9YdM/uM6z6PjGONUkWFNy6OnVidKDqQCSTIqkHzAHl7VpqV/ehecpkDDMa+91PX/\n9u48Pqrq/v/46zOTfd/DFghhSUAIEBJErGK1VkRrqeJa+9Pfo9Z+21pt1dYdFG21amv9/lxaa1u3\nVlCsFbW41FIXZEkICAGSICEkYcm+km2W8/vjDmGAQGImEwj5PB+PeTjLnXvPccx9e84995x2ZzvP\nbHqGF7e9yLSGaG77byRB20oIzcpi2OJFhKSn+1ys2pYOFq3Yyrub95E5KprHL5/GxORIn/erVH/S\nIPGiQTLEtTfBip/CtheFnc4AACAASURBVH/ChG/Cd/5otVJ6UFBTwH2r72Nn/Q5+uncqZ63YhWlu\nIe573yPhppuwR4T7XLR3N+/jvrcKaGl38vPzJ/KDs8Zq60SdNDRIvGiQKIyx1jd5/24IT4SFf4XR\np/f4NYfLwfNbnue5Lc8x3BnB/ZsnEPneWgISE0m+604i581DRHwqWk1LB/f9s4CVBfuZlhLDby/P\nZHyStk7UiadB4kWDRHXZu9Galr5pj7X64hmepX17UFxfzKLVi9hau5UrnVlc8XY9rsIdhM+ZQ/J9\n9xI81rf1TowxvL15H4veKqC108Vt50/khrPSsNt8CymlfKFB4kWDRB2mrQHe+gkUvgPp82HBMxDa\n85oiTreTl7e9zNObniaYQB6qPosRf1uF6egg7obvk/DDH2ILCfGpaFXN7dzzZgEfbqska3QMj10+\njXGJET7tU6m+0iDxokGijmIMrPsDfHAfRA6Hs35uhUpkz8vyljaWsvjzxeRX5XNe+ExuWh2JY+W/\nCRw1iuR77iby61/3sWiGtzbtZfGKrbQ7XPzignT+75ljtXWiBpwGiRcNEnVMFRvgnz+CmiLr9agc\nyLjYeiSMP+bX3MbNa0Wv8cSGJzAY7g1awGkvrKZzZwkR3ziPYXfdReDIkT4Vraqpnbvf3MK/t1eR\nPSaWxy6fxtgE3y/wK9VbGiReNEjUcRkDVduh8F0ofNsaMgyQkA4ZF8Gki2H4jG6HDe9t2cuSNUtY\nvXc12XHTubt0Kq6/LAVjSPjxj4m//jokKMiHohn+kb+HB97eSqfLzS8vyOD6OanYtHWiBoAGiRcN\nEvWVNJRD0b+sayilq8G4IHIEZMy3WiqpXwN7YNfmxhhW7FzBb3J/Q4ezg5+NvJa5/9jJgX9/RFBa\nGsMWLSJ8ds8jxI5nf2M7d/1jM6uKqpk1No7HFmYyJl5bJ8q/NEi8aJCoPmutg+L3rVD58iNwtlmT\nQk64wGqtjP8GBFsXw6tbq/nVul/xUdlHTI6fzAPmEgKefAFHRQVRF19M0i9/QWDS8WclPh5jDK9v\nqODBt7fhdBvump/BtaeP0daJ8hsNEi8aJKpfdLZCySqrC6xoJbTVgT0Yxn3dCpWJF2LCE/hg9wf8\net2vaepo4gfp13HpWmh4/i9IcDCJN99M7DVXIwEBfS7G3oY27vzHFj4prmZ2WhyPLZxGSlxYP1ZU\nKYsGiRcNEtXvXE4oW+O5rvIuNJaB2CBlNmRcREPaWfxmx1LeKXmH8THjWZLyI6Kffo0Dq1cTPGkS\nwxcvInT69D4f3hjDstxyHnp3O25juGv+JL47a7S2TlS/0iDxokGi/MoY2L/F6v4qfBcqC6z3k6fw\nyZjpPNC0hZrORr6XcS3XV6dT/+jvcFZWEnP55STe+nMCYnu+h+VY9jS0ccfyzXz2ZQ1njo/nN5dl\nMipWWyeqf2iQeNEgUQOqbtehlkrZGpoFfjcsheUhMDokkftn3sfoN3Kpe+kl7JGRJN1+G9GXXtrn\nmYWNMfx9fRm/fnc7IsI9F03iqpwUn6duUUqDxIsGiTphWqqh+D0ofIf1FZ+xOC6SisBArggaxk+i\nFtD0yme0bdxE6PTpDLt/MSEZGX0+VHldK3e8sZnPd9Zy1oQEfnNZJiNiQvuxMmqo0SDxokGiTgod\nLbQWr+SpLX/ilc69JLtcLKo7wNTayVT9pxpXSzux136XxJtvxh7Rt2lR3G7D39bt5uGVhdhFuPfi\nSVyRra0T1Te9DRK/zlctIvNEpEhEvhSRO7v5/HoRqRaRTZ7HDV6fPSoiW0Vku4j8rxzxlyAiK0Sk\nwJ/lV6pfBUcQNvVyfnnNB7w870XCIkfy46RoHh1bTuL5u4hJa6H+pZcp+cY5NL7+En35nzybTfje\nGam8d8vZTB4RxR1vbOH6v+ayr7HNDxVSyuK3IBERO/A0cCEwGbhaRCZ3s+kyY8x0z+N5z3fnAGcC\nmcAUIAeY67XvS4EWf5VdKX+bNmwmr1/2L27MvJGVoUF8Z8okCn54AalXhBMg9ey972HKvjmNjmX3\nQOU264L+VzA6PoxXfzCb+781mXW7avnmE5+wfENFn8JJqZ74s0UyC/jSGFNijOkElgLf7uV3DRAC\nBAHBQCBQCSAiEcCtwEP9XmKlBlCQPYifzvgpr178KkkRI7m1bg33nHs2EUv/TvJ3v0Z7lZOSB96g\n6kffxP3b6fD+PbB7Dbhdvdq/zSZcf+ZY3rvlbDKGRXL761/w/RfzqGxq93PN1FDjzyAZCZR7va7w\nvHeky0Rks4gsF5EUAGPMGmAVsM/zeN8Ys92z/YPAb4FWv5VcqQGUEZfB3y76G7dk3cLH5R+z4PNb\n+eyab5H274+JvvACardFUvK6ofmNv8Jf58Fv060VH4vfB0fPoZCaEM6yG8/gvosn8/nOGs7/3ce8\nsaECh8s9ALVTQ4HfLraLyOXABcaYGzyvvwfMMsb81GubeKDFGNMhIv8DXGGMOVdExgNPAld6Nv0Q\nuANoAh40xnxLRFKBd4wxU45x/BuBGwFGjx49c/fu3f6oplL9qqSxhMWrF7OpehNfG/k1Fs1eRNS2\ncvYvWULnlzuJmJlO8txQgmo+hc5mCIqwpmnJuBgmftOavuV4+69u4RfLN7Nhdz1hQXayU+M4fWwc\ns9PiyRwVTaAu86u8nPBRWyJyBnC/MeYCz+u7AIwxDx9jeztQZ4yJFpFfACHGmAc9ny0C2oFm4D6g\nEwgAkoDPjTHnHK8sOmpLDSYut4ulRUt5Mv9JBOHWmbeyMG0BDS+/QvVTT1szC994A3HnpGHb+R4U\n/gsOVIEtEMaeZU3Xkj4fokYcY/+GD7ft5/OdtawtqaW40rrcGBpoJzs11itYYggK0GAZyk6GIAkA\nioHzgD1ALnCNMWar1zbDjTH7PM+/A9xhjJktIlcCPwDmAQK8B/zeGPO213dTOU6LxJsGiRqMKpor\nuH/N/azbt47s5GwemPMAww8EUvnwIzR/8AFBqakMW3Qf4bNnw54868767e9A3U5rByOzrVDJuBgS\nJx7zOLUtHazfVcfaklrW7aqjcH8zACGBNmaOieX0sfHMTotnWko0wQH2gai6Okmc8CDxFGI+8HvA\nDvzFGPMrEVkC5BljVojIw8AlgBOoA35kjCn0tE6eAc7GuvD+njHm1iP2nYoGiTrFGWN488s3eTz3\ncRxuBzfNuIlrJ11L22er2f/Qr3CUlRE1fz5Jd9xBYHKSNbqrusgzXcs71hr1APETrHVVMi6GEVnd\nrq1yUN2BzsOCZfu+JgCCA2xkjY5ldlo8p6fFMT0lhpBADZZT2UkRJCcLDRI12FUeqOShtQ/x34r/\nMjVhKkvmLCEtLIXaPz1P7XPPIYGBJN5yM7HXXHP4zMKNFdZMxdvfhtLPrLVVIoZZLZXTfwiJ6T0e\nu6H1YLBY4bJ9fxPGQFCAjRkpMV3BkjU6VoPlFKNB4kWDRJ0KjDGs3LWSR9Y/QrOjmRszb+SGKTdg\nKvay/6FfceDTTwnOyGDYokWEZc04egdt9VD8gdVS2fEhONthyqVw9i8hqfdTszS2OlhfWse6klrW\n7qpl294m3AaC7Damj45htucay4zRsYQGabAMZhokXjRI1Kmkrr2OR9Y9wsrSlUyMnciSOUuYHD+Z\n5g8/pPLXD+Pcv5/ohZeRdNttx55Z+EANfP7/YP2fwNEKpy2wAiW5u3uGj6+xzUFeaR3rPN1hBXsa\ncRsItAvTU2K6rrFkjYkhLKjv67CogadB4kWDRJ2KVpWt4sG1D1LbXst1p13Hj6f9mKAOF9XPPEPd\niy9hDw8n8bZbiVm48NgzCx+ohTVPwfrnoLMFJi+AuXf0KVAOamp3sKG0nrUltazdVUfBnkZcbkOA\nTZiWEtM1KmzmmFjCgzVYTmYaJF40SNSpqqmzid/m/ZZ/7PgHqVGpPDDnAbKSs2gvLmb/kiW05W0g\nZFomwxcvJmTyccKhtc4KlHXPWfenTP62J1BO87mMLR1O8kqtayzrdtWyueJQsEwdFW1dYxkbR3Zq\nHBEaLCcVDRIvGiTqVLdm7xoeWPMAe1v2clXGVfws62eEBoTStGIFlY8+hqu+nthrriHxlpuxR0Ye\ne0etdbD2GVj7BytQJn3LCpRhU/utrAc6nGzYXd81KuyL8gacboPdJkwZGc3stDhmj40nOzWWyJDA\nfjuu+uo0SLxokKihoNXRypP5T/Jq4asMDx/O4jmLmTNiDq7GRqqffJL6V5diT4gn+Zd3EHXxRcef\nWr61DtY+C+v+AB1N1rDhuXfA8Mz+L3enk/zdDVZXWEktX1Q04HAZbIInWOKZnWa1WKI0WAaUBokX\nDRI1lGys2sii1YsobSplwfgF3J59O9HB0bRtKWD/kiW0b9lC2KxZRH/7EsJycghMOc56JW31Vutk\n7bPQ0QjpF8E5d8DwaX4rf1uni/yyemtUWEkdm8ob6HS5sQmcNiK66xpLztg4okM1WPxJg8SLBoka\najpcHTy76Vle2PoCcSFx3Dv7Xs4dfS7G5aLh9depfuppXDU1AAQkJRGWk0NYTjZh2dkEjRt3dLC0\nNVitk7XPQHujNQXL3F/CiG6GGfezdsfBYLFGhW0sb6DT6UYEJg+P8owKi2PW2DhiwoL8Xp6hRIPE\niwaJGqq21m5l0epFFNcXMy91HnfOupP40HiMMXTu3ElrXh6t63Npzc3FWV0NgD02lrDs7K5wCZ44\nEbF77gdpb4R1f7QuzLc3wsR5VpfXyKwBq1O7w8WmcqsrbF1JHfll9XR4giVjWFRXi+X0sXHEhmuw\n+EKDxIsGiRrKHC4Hfy74M3/c/EciAiO4c9adzB87/7BWhzEGR1mZFSy5ebTm5uLYswcAW1QUYVlZ\nVoslJ4eQSZMQV6s1wmvNU9DeABMusLq8Rs4c8Pp1OF18Ud7ouXhfy4bd9bQ7rCnyM4ZFHgqWtHji\nNFi+Eg0SLxokSsGX9V+y+PPFbK7ZzFkjz+LCsReSHpfO2OixBNqOvtbg2Lv3ULDk5dG5axcAEhZG\n2PTphM3KISxzEiFta7HlPWtdTxl/PpxzJ4zq8dzjN51ON5srGrpGheWV1tPmcGETOCc9iStzUjg3\nI0mnzO8FDRIvGiRKWVxuF69sf4VnNj1Dq9NaGy7QFsj4mPFkxGWQHpdORlwGE2MnEhl0+DBhZ3U1\nrRs2WF1heXl0FBcDIEFBhGZOIWyYIcyxjtDIWmwZ58HcOyElZ8DreKROp5stexr5T2ElyzdUUNnU\nQUJEMJfNHMmV2SmkJUac6CKetDRIvGiQKHU4p9vJ7qbdFNYVUlRXRGFdIYV1hdR31HdtMzJi5KFw\nic0gIy6DYeHDurrEnPX1tOXnd3WFtW/fDm432IXQOBdhCQcIy5xM6NV3Y8845wTV9HBOl5uPi6tZ\nllvOR4VVuNyGWalxXJmTwvypw3VusCNokHjRIFGqZ8YYqtuqDwuXovoiyprKMFjniaigqMNaLumx\n6aTFpBFoC8TV0uIVLOto21IALjeIISQ5hLA5cwk791uEzpx57DnABlBVcztvbNjDa3nl7Ko5QGRw\nAJdMH8FVOaOZMjLq+PfZDBEaJF40SJTquwOOA+yo32GFS70VMsX1xXS4OoBDXWPe3WLpcelEuAJp\ny11L69t/pjU3l7Zqwbisk3PwhAldF+/DsrMJSEw8YfUzxrB+Vx3Lcsv5V8E+2h1uJg+P4sqcFBZM\nH0l02NC9V0WDxIsGiVL9y+l2UtZUZnWJ1R9qwdS113VtMzJiJOmxnpZLVCrpJXnErHyJtrJWWluS\nad3nxrRbYRSUmtp1H0tYTg6BI7pfJtjfGtscrPhiL8tyyyjY00RQgI35U4ZxRU4Ks8fGY7MNrVaK\nBokXDRKl/M8YQ01bTVeX2MFw2d20u6trLDIwkoyASNJry8hobiTdOYG4wK/RUVJP64YNuJus1RgD\nR4w47CbJwDFjBryrqWBPI6/llfPmxj00tzsZEx/GFdkpLJw5iuSokAEty4miQeJFg0SpE6fV0cqO\nhh2HrrvUFVFcX0S7p2sswBjGEURGUhbTTQbjSzuI3r4HR/4XuOqsFk5AYuJhXWFB48Yde2r8ftbu\ncPFewX6W5paxtqQOm8C5GUlckZ3C10/xYcQaJF40SJQ6ubjcLnY376aoaguFhW9QtH8DRTZDTcCh\nUVMjwoZzeucosvaFkLKzifCCUkx1LQD2mJjDusKC09MP3X3vR6U1B3gtr5zXN1RQ3dxBYmQwC2eO\n4orsFMYmhPv9+ANNg8SLBolSJzlHG+S/RM3qJyhy1FOYNJ6i5PEUdtZT2lhqdY0ZQ1pLOGfXJjCl\nwsaw4lqCKq3hyraICEJnZhHuabGEnHYaEui/i+ROl5tVRdYw4lVF1jDi08daw4gvnHLqDCPWIPGi\nQaLUIOFoh/yX4LMnoHkvpMym9ayf82XMcAoPXnepL2RH/Q7anG3ENxmmlNuYVRnOxN0uovc3AyAh\nIYRlzSA0O5vwnBxCMjOxBQf7pchVTe0sz69gWW45u2tbiQwJYMH0kVyZk8KUkdEYY8DlArfbeu52\nH/3c+59HbuNyHfb+Uc+7+a71vgHjJuz00/vcDahB4kWDRKlBxtEOG1+2AqVpD6Scbk0OOe5cEMHl\ndlHeXH7YiLGiuiI6a6rIKDdMLjNk7glgRKUDMWACA7BPGEdwSLjnBO050Xo/d7m6f//IE3Z3YeBy\nYYzB7XLhdlnbizHYOfHn1/QvNvU5RDVIvGiQKDVIOTusQPn0CWiqgFE51lxe486DbkZx1bTVUFxX\nTGG9dad+WcU2wrbtJqPMxZgqCLIFEhEcRWRIFFHB0YQHR2Kz2cFuQ8QGNhvY5Ojndnv37x98brOD\nzYbYBMRGh8tQXH2Agr3NVB7oxGa3M2FYFJkpMaTER1gtBK/tsdus9+Tg/g4+92xjsx/xvnTtAxHr\n+tBR37Weh86Y0efrRxokXjRIlBrknB2w6W/w6e+gsRxGZluBMv4b3QaKtzZnGzsbdrK1Ziv5Vfls\nrNrIvgP7AAgPDGda4jRmJM0gKymLKQlTCAsM69eiF+xpZGluGW9t2ktzu5PU+DCuyElhYdYokk7y\nYcQaJF40SJQ6RTg7vQKlzJq2fu6dMOH8HgPF276WfWys2tgVLDvqd2AwBEgAk+IndQXLjOQZxIXE\n9UvR2zpdrCzYx9LcctbvqsNuE76ensRVOSmck55IwEk4jFiDxIsGiVKnGGcnfPEqfPo4NJRZKzXO\nvRMmXvCVAuWgps4mNlVtssKlMp+CmgI63Z0ApEalkpWc1RUuKZHHWZq4l0qqW3gtr4LlGyqoaekg\nyWsYcepJNIxYg8SLBolSpyiXwwqUTx6Hht0wfLrV5TVxXp8C5aBOVyfbardZLZZKq+XS1GnddR8f\nEk9WclZXiyU9Np0AW0CfjuNwuVlVWNU1jNhtYHZaHFfljGbelGGEBJ7YYcQaJF40SJQ6xbkcsHkZ\nfPIY1JfC8GnWKK/0+T4FykFu46akoaSrK2xj1Ub2tFgrSIYGhDItcVpXsGQmZPbpOsv+xnbe8Awj\nLqtrJSokgAUzrGHEp42I9rkOfaFB4kWDRKkhwuWAza95AmUXDMu0AiXjon4JFG/7D+xnU9WmrnAp\nqivCYLCLnUlxk5iRbHWFTU+aTkJoQq/363Yb1u6qZdn6Mj7Yuhe300HmiDAWzhjBhZMTiAoScDut\nurpd1vPjPdLOtUZ39YEGiRcNEqWGGJcTtrwOnzwKdSWQPBXm3AShsZ4T8MET7cETseOI1709Ubu6\n9tfs6uALVzP5rmY2mgNsoYMOz30kY4ydGS47WS4bWQ7DaJcL6Xbfh/aHcfXPv4t7KiGwb6PDNEi8\naJAoNUS5nFCw3Gqh1H7Z9/3YAsEW4HnYwX7Ea1uA1zbWa4fNzja7YaN0kk8HG2mnASsc4gggKyCK\nGQExZAXGkh4US6A96Jj7M7YA9jU7yCtvZlNFCy1OiIsIIzstkZxxSUSHhR4qjz3Aq2wB1sg2m95H\n4jMNEqWGOLcL9n0BGK+TbKDXSfvgSbib9/p4Ej6qCMZNaWNpV1dYfmU+FS0VgHWdJTMx07rOkjSD\naYnTjnmdpbXTyb+27Oe13HLWl1rDiM/LSOLKnBTmTuzfYcQaJF40SJRSJ6Oq1qqui/f5lfkU1Rfh\nNm7sYic9Lr0rWLKSs7q9zrKzuoXXcst5I7+CmpZOkqMODSMeE+/7MGINEi8aJEqpwaCls4XN1Zu7\nWi2bqzfT7moHYHTk6K5QmZE0g9So1K77WRwuNx9tr2JZbhkfF1fjNjBnXDxX5qQwb8owggO0a8tn\nGiRKqcHI4XZQWFtIflU++ZVWuNR3WFPnxwbHdgVLVlIWGfEZBNoC2dfYxvK8CpbllVPV3MH6u88j\nJiyoT8fXIPGiQaKUOhUYYyhtKu3qCttYtZGy5jIAQuwhZCZmHpo3LH4q+xthYnJkn4+nQeJFg0Qp\ndaqqaavpCpb8qnwK6wpxGzc2sZEem85z5z9HTEhMn/bd2yDp2339SimlTgoJoQmcP+Z8zh9zPgAH\nHAfYXL2ZjVUbKa4vJjrY/3fF+zVIRGQe8CRgB543xjxyxOfXA48BezxvPWWMed7z2aPARYAN+BC4\nBQgFXgfGAS7gbWPMnf6sg1JKDSbhgeGcMeIMzhhxxoAd02/zFouIHXgauBCYDFwtIpO72XSZMWa6\n53EwROYAZwKZwBQgB5jr2f5xY0wGMAM4U0Qu9FcdlFJK9cyfE+DPAr40xpQYYzqBpcC3e/ldA4QA\nQUAwEAhUGmNajTGrADz7zAdG9XvJlVJK9Zo/g2QkUO71usLz3pEuE5HNIrJcRFIAjDFrgFXAPs/j\nfWPMdu8viUgM8C3gI38UXimlVO/4M0i6m2rzyCFibwOpxphM4N/AiwAiMh6YhNXaGAmcKyJnd+1Y\nJAB4FfhfY0xJtwcXuVFE8kQkr7q62ufKKKWU6p4/g6QCSPF6PQrY672BMabWGNPhefknYKbn+XeA\ntcaYFmNMC7ASmO311eeAHcaY3x/r4MaY54wx2caY7MTERB+ropRS6lj8GSS5wAQRGSsiQcBVwArv\nDURkuNfLS4CD3VdlwFwRCRCRQKwL7ds933kIiAZ+5seyK6WU6iW/Df81xjhF5Cbgfazhv38xxmwV\nkSVAnjFmBXCziFwCOIE64HrP15cD5wJbsLrD3jPGvC0io4B7gEIg3zPPTNeQYaWUUgNP72xXSinV\nLZ0ixYuIVAO7+/j1BKCmH4szGGidh4ahVuehVl/wvc5jjDE9XmQeEkHiCxHJ600in0q0zkPDUKvz\nUKsvDFyd/XmxXSml1BCgQaKUUsonGiQ9e+5EF+AE0DoPDUOtzkOtvjBAddZrJEoppXyiLRKllFI+\n0SDxEJF5IlIkIl+KyFFrnIjI/4jIFhHZJCKfHWNK/EGlpzp7bbdQRIyIDOoRL734ja8XkWrPb7xJ\nRG44EeXsT735jUXkChHZJiJbReTvA13G/taL3/kJr9+4WEQaTkQ5+1Mv6jxaRFaJyEbPJLnz+7UA\nxpgh/8C6834nkIY1df0XwOQjtonyen4J1t32J7zs/qyzZ7tI4BNgLZB9osvt59/4eqyZEk54eQew\nzhOAjUCs53XSiS63v+t8xPY/xZp144SX3c+/83PAjzzPJwOl/VkGbZFYelw7xRjT5PUynKNnMh5s\nertezIPAo0D7QBbOD3xZH2ew6k2dfwA8bYypBzDGVA1wGfvbV/2dr8aaSXww602dDRDleR7NERPo\n+kqDxNKrtVNE5CcishPrxHrzAJXNX3qss4jMAFKMMe8MZMH8pM/r4wxivanzRGCiiKwWkbWe5bEH\ns97+zojIGGAs8J8BKJc/9abO9wPXikgF8C+slli/0SCx9GbtFIwxTxtjxgF3APf6vVT+ddw6i4gN\neAK4bcBK5F99Xh9nEOtNnQOwurfOwfq/8+c9i8YNVr36W/a4ClhujHH5sTwDoTd1vhp4wRgzCpgP\nvOz5G+8XGiSWHtdOOcJSYIFfS+R/PdU5EpgC/FdESrHWg1kxiC+4+7I+zmDVm/+uK4C3jDEOY8wu\noAgrWAarr/K3fBWDv1sLelfn7wOvQdcKtCFY83D1Cw0SS2/WTvH+47oI2DGA5fOH49bZGNNojEkw\nxqQaY1KxLrZfYowZrNMo+7I+zmDVY52BfwJfBxCRBKyurm5XHR0kelNnRCQdiAXWDHD5/KE3dS4D\nzgMQkUlYQdJvS8f6bT2SwcT0bu2Um0TkG4ADqAeuO3El9l0v63zK6GV9j7U+zqDUyzq/D3xTRLYB\nLuAXxpjaE1dq33yF/66vBpYazzCmwayXdb4N+JOI/Byr2+v6/qy73tmulFLKJ9q1pZRSyicaJEop\npXyiQaKUUsonGiRKKaV8okGilFLKJxokSvlIRO4XkdtPgnKUeu4FUWpAaZAopZTyiQaJUt0QkXAR\neVdEvhCRAhG50vv/+EUkW0T+6/WVaSLyHxHZISI/8GwzXEQ+8ax7USAiZ3nef1ZE8jzrfzzgdcxS\nEfm1iKzxfJ4lIu+LyE4R+R/PNud49vmmZw2RP3Q3Z5KIXCsi6z3H/qOI2P3570sNbRokSnVvHrDX\nGDPNGDMFeK+H7TOxps45A1gkIiOAa4D3jTHTgWnAJs+29xhjsj3fmSsimV77KTfGnAF8CrwALMSa\n52yJ1zazsO5UngqMAy71LohnCowrgTM9x3YB3/0KdVfqK9EpUpTq3hbgcRH5DfCOMeZTke4mWe3y\nljGmDWgTkVVYJ/tc4C8iEgj80xhzMEiuEJEbsf7+hmMtNLTZ89nBKTy2ABHGmGagWUTavWblXW+M\nKQEQkVeBrwHLvcpyHtaEk7meMocCg32dEXUS0yBRqhvGmGIRmYk15fbDIvIB1hxcB1vxIUd+5ehd\nmE9E5GyslsrLIvIYVkvjdiDHGFMvIi8csa+Dsw+7vZ4ffH3w7/WoYx3xWoAXjTF39VBNpfqFdm0p\n1Q1P11SrMeYV6KnVgQAAAPVJREFU4HEgCyjl0NTylx3xlW+LSIiIxGOt7ZHrWTipyhjzJ+DPnn1E\nAQeARhFJBi7sQ/FmeWZ6tWF1YX12xOcfAQtFJMlTlzhPWZTyC22RKNW9qcBjIuLGmvH5R1hdRH8W\nkbuBdUdsvx54FxgNPGiM2Ssi1wG/EBEH0AL8H2PMLhHZCGzFmq59dR/KtgZ4xFPGT4A3vT80xmwT\nkXuBDzxh4wB+Auzuw7GU6pHO/qvUICIi5wC3G2MuPtFlUeog7dpSSinlE22RKKWU8om2SJRSSvlE\ng0QppZRPNEiUUkr5RINEKaWUTzRIlFJK+USDRCmllE/+P/j8BxwZXzB0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1efd1312278>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# summarize results\n",
    "print(\"Best: %f using %s\" % (gsearch.best_score_, gsearch.best_params_))\n",
    "test_means = gsearch.cv_results_[ 'mean_test_score' ]\n",
    "test_stds = gsearch.cv_results_[ 'std_test_score' ]\n",
    "train_means = gsearch.cv_results_[ 'mean_train_score' ]\n",
    "train_stds = gsearch.cv_results_[ 'std_train_score' ]\n",
    "\n",
    "pd.DataFrame(gsearch.cv_results_).to_csv('my_preds_subsampleh_colsample_bytree_1.csv')\n",
    "\n",
    "# plot results\n",
    "test_scores = np.array(test_means).reshape(len(colsample_bytree), len(subsample))\n",
    "train_scores = np.array(train_means).reshape(len(colsample_bytree), len(subsample))\n",
    "\n",
    "for i, value in enumerate(colsample_bytree):\n",
    "    pyplot.plot(subsample, -test_scores[i], label= 'test_colsample_bytree:'   + str(value))\n",
    "#for i, value in enumerate(min_child_weight):\n",
    "#    pyplot.plot(max_depth, train_scores[i], label= 'train_min_child_weight:'   + str(value))\n",
    "    \n",
    "pyplot.legend()\n",
    "pyplot.xlabel( 'subsample' )                                                                                                      \n",
    "pyplot.ylabel( 'Log Loss' )\n",
    "pyplot.savefig( 'subsample_vs_colsample_bytree1.png' )"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
